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Abstract — An enhanced covering lemma for a 
Markov chain is proved in this paper, and then the 
distributed source coding problem of correlated 
general sources with one average distortion cri- 
terion under fixed-length coding is investigated. 
Based on the enhanced lemma, a sufficient and 
necessary condition for determining the achiev- 
ability of rate-distortion triples is given. 

I. Introduction 

In the classic problem of multiterminal source coding, 
M (M > 2) correlated general sources have to be com- 
pressed separately from each other in a lossy fashion, i.e., 
with respect to a fidelity criterion, and then decoded by 
the common decoder which has access to a side infor- 
mation source that is correlated with the sources to be 
compressed. This situation is illustrated in Fig. ^ an d it 
is also called distributed source coding. The well-known 
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Figure 1: Separate compression of M correlated general 
sources with side information at the decoder 

Slepian-Wolf coding problem and the Wyner-Ziv coding 
problem can be regarded as two special cases of this sit- 
uation. These two special cases were solved in 1970's for 
stationary memoryless sources [1,2], and later extended 
to the case of general sources [3,4]. However, for this gen- 
eral problem, no conclusive results are available to date. 
Even for the special case that the sources are memory- 
less and stationary and the distortion measure is additive, 
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only inner and outer bounds are derived in [5,6], etc. Re- 
cently, in [7] , we adopt an information-spectrum approach 
to solve this open problem for general sources with maxi- 
mum distortion criterions under fixed-length coding and a 
general formula for the rate-distortion region is obtained. 
Though the formula in [7] is incomputable in general and 
can not be used to obtain the single letter rate-distortion 
region for correlated memoryless sources, it does provide 
a very general sufficient condition, which includes many 
previous results (e.g. in [4,5]) as its special cases. 

In this paper, we goes further to investigate the prob- 
lem with average distortion criterions under fixed-length 
coding. Since the covering lemma for a Markov chain 
plays an important role in the proofs of these kinds of 
problems, we established an enhanced covering lemma in 
Section [HJ which is the main contribution of this paper. 
Then in Section ITTT1 we investigate the distributed source 
coding problem of two correlated general sources with one 
average distortion criterion under fixed-length coding. 

II. Covering Lemma 
In this section, we will prove a covering lemma in a 
very general form. For comparison, the original covering 
lemma in [4] is first shown below. 

Lemma 1 (Lemma 1 in [4]) Let U n , V n and W n be 

random variables which take values in finite sets U n , V™ 
and W n , respectively, and satisfy a Markov condition 



Pun 



U n V n W 



•Pi 



U n \V*> 



for each n. Now let {^ n }^Li be a sequence of arbitrary 
functions such that 

V n :U n x W n {0,1}, 
lim Pr{V n (U n ,W n ) = 1} = 0. 

n — >oo 

Then, for any 7 > 0, there exists a sequence {-FnlJ^Li °f 
function F n : V™ — ► {wi}f^[ n W™ such that 

X *-L n 

lim Pr{* n (C/",F„Cl/' l )) = l} = 0. 

n — *oo 

The quantity Z(V; W) in Lemma ^ is called the spec- 
tral sup-mutual information rate in the methods of 
information-spectrum [8] , and it is defined by 

I(V; W) = p- km sup - In — — — , (1) 

n-*oo n Pv"{V n )Pw^{yy ) 
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where V and W denote the random sequences {V n }^ =1 
and {W 1 }^! which are called general sources in the 
methods of information-spectrum, and 



p- lim sup Z n = inf < a 



lim Pr{Z n > a} = 



denotes the limit superior in probability of the sequence 
{Z n }^L 1 of real-valued random variables. 

Now, let us establish the enhanced covering lemma. 

Lemma 2 Let U n , V n and W n be random variables which 
take values infinite setsU n , V n andW n , respectively, and 
satisfy a Markov condition 



Pi 



u n v n w„ = p u n v n p w n \v n 

for each n. Now let {A n }n%i be a sequence of arbitrary 
sets in U n x W n satisfying 



lim Pr{(U n ,W n ) € A n } = 1, 



(2) 



and let {d n }n%i be a sequence of arbitrary functions d rt 
U n x W n — ► [0, oo) satisfying 



Do = supmaxjlm d n } < oo, 



(3) 



where \md n denotes the image of d n , then for any 7 > 0, 
there exits a sequence {Fn}^! °f random functions F n '■ 
V n — > W n such that 



\F n {Vn)\ < 



(4) 



lim Pr{(t/ n , F n {V n )) eA n } = l, (5) 
\imsup(E[dn{Un,F n (Vn))} - E[d n (U n ,W n )]) < 0, (6) 

where V = {V n }%L x> W = {W n }™ =1 . 
Proof: Let us define 

(11/ x A 



TJ n 1) (v,w)= ^ P U n \V n W r . 

ueu„ 



(u\v,w)l{(u,w) A n } 



]T Pu n \v n (u\v)l{(u,w) # A n } (7) 



uEU n 



and 



VnH v , w ) = Y P U n \V n W n { U \ V ' W ) d n(u,w) 



E P U n \V n ( u \ v ) d n{u,w) 



(8) 



ueu n 



for v e V n , w e W n - Then, it follows from J5J) and 
that 



lim E[r£\V n ,W n )]=0, 

n— >oo 

max r)W(v,w) < D , 

d6V„,ju6W„ 

E[ri£\Vn,W n )] = E[d n (U n ,W n )]. 



We denote E[rjn\v n , W n )} by S n , and define the set 

T« = {{v, w) e V„ x W n \r&\v, w) < V^}- (9) 
Clearly, by Markov's inequality, we have 

Pr{(V n ,W n ) ^TW} < ^{Vn^Wn)] = ^- (10) 

Letting p be an arbitrary nonnegative real numbers, we 
define 



T { n\p) = I («,!») G V„ X W r . 

Next, set 



1 , p w n \v n {w\v) 

— In ; ; < p 



n P Wn (w) 



M„ 



(11) 
(12) 



We generate a sequence W^f n = {W n ,i}fitl, eacn m " 
dependently subject to the identical probability distri- 
bution Pw„ ■ Now, let us define the random function 
F n ■ V n —> W n with respect to the random sequence 
W^f n . For any v £ V n , define 

p / \ A f ^min(v), 5„(^,iy„ M ") ^0 (13a) 

n[V) I S„(u,VKf")=0 (13b) 

where 

w min (v) = arg min rffi{v,w), 

«4h»=arg min («,«>), 

S>,™ M ») ^ { Wi }^i n{we W n \(v,w) e T«}. 

Clearly, F n satisfies the requirement @. Next, let us 
estimate the upper bound of Pr{(t/ n , F n {V n )) £ A n } and 
£[d„([/„,F n (K))]. First, we have 

Pr{(U n ,F n (V n )) £A n } 

= E *v» E ^»™(^ M ") 

E JV B |y B («|«)1{(«, /»(»)) £ A n } 



»ev, 



= E^» E V^tf'^./nW) 

/nW) G T«} + 1{(V, f n (v)) 4 T«}) 

(b) 



< \f^+ E 
vev n 

P W M,(w M ~)l{{v,fn{v))tT^} 



w Mn G VV„ n 



C+ E P v n (v)M(v,F n (v))?TW}, 
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where (a) follows from Q, and (b) follows from @. Fur- 
thermore, we have 

Pr{(v,F n (v))?TP} 
( = E ^,f"(^ M ")l{^(^^ M ")=0} 

(b) 



= E *V„(«0l{(tMi0£TW} 



= (i- E iV.WU^^eTW}] 

^ wew, 



M„ 



< (i- E iV.Wi^.^e^n^W} 



M„ 



( c ) / x , 

l{(«,«;)GrWnTW(p)}' 
< 1- E *V»|v„H«0 

where (a) and (b) follows from Ijl3(l , (c) follows from , 
and (d) follows from the inequality (1— xy) n < l—x+e~ yn 
for < x, y < 1, n > 1. Then we have 

Pr{(C/ n ,T„(14)) g A„} 
= vC+ E P v;»(l- E p w n \v n (w\v) 



-M„ 



vev„ 



i«ew„ 



-M„ e -" p 



l{(v,w)eT^nTi 2 \p)} + e 

= \fa + Pr{(K, W„) £ T« n Ti 2 )(p)} + e- M ^ n " 
< v^ + Pr{(V n ,W„) gT«} 

+ Pr{(K,^„)^T( 2 )(p)} + e -^" P 



(a) 



-M„e-" p 



< + Pr{(K, W„) £ T^fc)} + e 

where (a) follows from jESjl - Letting p = /(V; W) + ^, 

we have 

Pr{(£/„,F„(K)) £AJ 
< 2^ + Pr{(V n ,W n ) # r«(7(V; W) + J)} 



(6) 







as 7i — ► oo , where (a) follows from (|12|l . and (b) follows 
from (JIJ and IpTjl . This concludes 

Second, the expectation E[d n {U n , F n (V n ))] can be 
written as 

£[d„([4,F„(K))] 



E^v» E p w^(w M ") 

vev n w M n e w*'" 

E P Un\V n (u\v)d n (u, f n (v)) 



(a) 



E^v» E ^(^) 



/•■Do 

= E *V» / Q(v,p)dp, 

where (a) follows from (JSJ, and 

Q(«, /3) = E P w,f- (™ M ")lR 2) (*>, /nW) > /?}• 
Furthermore, we have 



Q(v,0) 



(a) 



M„ 



^ e n^w^^)^ 



or («,«ii)0T«} 



ii/„ 



= (i - Y, p w n MHv ( nHv,w) < (3, (v,w) e rW}] 
< fi- E %Wi{ii 2) KH<ft 



M„ 



(b) 



< (l-e- n " E Pw n \vJw\v)l{ni 2 \v,w) < /?, 
u>ew„ 

(«,«;) GTWnT«(p)} 



< 1- E p w„|y„(^|«)lM 2) (-",^) < A 

(,») € T«nf(ri} + e-^ 
E P Wn |v n (^Wl{r?4 2 ) (t7,io) > /3 or £ T« 



or (v,w)?TW{p)} + e- M ^ 



where (a) follows from (|13|l . (b) follows from Ijlip. and (c) 
also follows from the inequality (1 — xy) n < 1 — x + er yn 
for < x, y < 1, n > 1. Then we have 



£[d„(t/ n ,F n (y n ))] 

rD 



< E p k» / °( E p ^w, 

l^foto) >0or (v,w) £ T « 
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(,,H^( 2 )(p)} + e - Jf ^j 



dp 



l{ Vn 2 \v,w)>(3} 



< X! p v n w n {v,w) 

+ l{(v,w)?TW} + l{(v.iv> r T'J'i,,)}),!! 

+ D e- M ^ P 



vev n ,wew n 



+ D l{(v, w) # T\ } + D l{(v,w) # TW(j>)} 

( = } E[d n (U n , W n )] + D a Pr{{V n , W n ) £ T«} 
+ D Pr{(V„, W n ) £ Ti 2 \p)} + D e- M - e ~ n " 

where (a) follows from (jHJ. Letting p = I(V; W) + ^ 
have 



E[d n (U n ,F n {V n ))} - E[d n {U n ,W n )] 

+ D Q Pr{(V n ,W n ) ? T^(I(V;W) 

>> - 

L>oe 



(a) ,- 



(b) 







as n — > oo, where (a) follows from (|10|l and (|12|) . and (b) 
follows from and JTTJ. This concludes and hence 
completes the proof. □ 

Remark 1 The main idea of this proof is a combination 
of the ideas in the proofs of Lemma 1 in [4] and Theorem 
5.5.1 in [8]. However, such a method has its own lim- 
itation. Because the minimum operation in \Y6\ should 
be applied to an ordered set, we can establish covering 
lemmas with only one average distortion criterion. 

By Lemma[2J we can easily obtain the following propo- 
sition that is also a generalized version of Lemma 1 in [4] . 

Proposition 1 (Lemma 2 in [7]) Let U n , V n and W n 

be random variables which take values in finite sets U n , 
V n and W n , respectively, and satisfy a Markov condition 

Pu n v n w n = Pu„V n Pw n \v n 

for each n. Now let {B n } n < ' =1 be a sequence of arbitrary 
sets in U n x W„ satisfying 



liminfPr{(^ n ,W r n )€fl n } = e, 



(14) 



then for any 7 > 0, there exits a sequence {Tijj^Lj of 
random functions F n : V n — ► W n such that 



\F n (V n )\ < 



o n(/(V;W0+7) 



liminf Pr{([/„,F„(K)) € B n } > e. 



(15) 
(16) 



Proof: Letting A n = U n x W n and d n = l{(u,w) £ 
B n } and then applying Lemma [21 we have 

limsup(E[d n (U n ,F n (V n )} - E[d n (U n ,W n )}) < 0, 

n — >oo 

where F n is the random function constructed in the proof 
of Lemma |5J Then we have 

liminfPr{(;y„,F„(V4)) G B n } 

n— >oo 

= 1 - limsup E[d n (U n ,F n (V n ))} 

n — *-oo 

>1- [\imsup E[d n (U n ,W n )} 

n — *oo 

+ limsup(E[d n (U n ,F n (V n ))] - E[d n {U n ,W n )])] 

71 — >00 

> l-limsup£J[d n ({7 n ,W n )] 
= \imMPr{(U n ,W n ) £ B n }. 

n — >oc 

This proves the proposition. □ 

III. MULTITERMINAL SOURCE CODING 

In this section, we will investigate the sufficient and 
necessary condition for multiterminal source coding with 
one average distortion criterion. 

As depicted in Figure ^ a multiterminal source cod- 
ing system can be stated as follows. For simplicity, we 
will only consider the case of two terminals without side 
information at the decoder. Given a pair of correlated 
general sources {Xi,X 2 ) with finite alphabet X\ X 
each characterized by an infinite sequence 

X m = {Xl\ = (iJUg, • • • , Xf&)}~ ls m = 1, 2 

of n-dimensional random variables X^ taking values in 
the n-th Cartesian product X™, the n-length source out- 
puts {X™, X%) are separately encoded into a pair of fixed- 
length codewords (<$:\x? ), <j)£\x%)), and then the 
common decoder observes these codewords to reproduce 
the estimates (Yp,Yp) = (^(X? , X 2 "), ^ ] (Xf, X?)) 
of (X", X%). Here, the pair of encoders are maps defined 

by 

^■.X«^I L w "{l,2,..,4 ra) }, 
and the rate of each encoder is calculated by 



(m) a ln|0j m) (^)| 



The decoders f/'n"'' (jn = 1,2) are maps defined by 



y r ' 



where y„\ denotes the n-th Cartesian product space in 
which the n-dimensional estimate Y£ takes values. Next, 
let us define the distortion measure. A general dis- 
tortion measure d is a sequence of functions 
d n : X[ l x X% x x y% -> [0,+oo), and hence the 
average distortion is calculated by 

E[d n {X?,X%,Y?,Y?)}. 
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Then for such a system with an average distortion cri- 
terion, a rate-distortion triple (Ri,R2,D) is said to be 
/a-achievable if and only if there exists a sequence of 
fixed-length codes such that 

lim sup R(4>^ ) < Ri , lim sup R{^ ] ) < R 2 , 

lim sup E [d n (X?, x$, y™, y 2 ™)] < D. 

n — >oo 

Therefore, our task is to find the set of all the triples 
(Ri, R 2 , D) that are /a-achievable. By applying Lemma 
we proved the following sufficient and necessary condi- 
tion. 

Theorem 1 For a pair of correlated general sources 
(Xi, X2) and a distortion measure d satisfying the con- 
dition (0, the rate- distortion triple (R\, R 2 , D) is fa- 
achievable if and only if there exist a pair of random se- 
quences (Z^\Z^) = {(Z { n 1} ,Z ( n ] )}n=i with the alpha- 
bet {Zn x -Zl^l^-L and a pair of function sequences 
(hW,hW) = Ag )}^! defined by 



h(m) . Z (l) z (2) -yr, 



Lemma 3 Let X", Xg, Z„ \ and Z n 2 ^ be random vari- 
ables which take values in finite sets X" X 2 , z[\^ and 
Zn\ respectively, and satisfy 



P 



for each n. Now let {v4. n }^i be a sequence of arbitrary 
sets in X" x X% x Zn^ x Z n 2 ' satisfying 

lim Pr{(X[\X-,Z^,Z^)eA n } = l, 

n — >oo 

and let {d n }^ =l be a sequence of arbitrary functions d n : 
X" x X2 x Zn x Z n 2 ^ — > [0,oo) satisfying the condition 
J3J|, then for any 71,72 > 0, there exit two sequences 
{i^ m) },? = i of functions F ( n m) : X r \\ 
such that 



Z n m) (m = 1,2) 



n(J(X m ;Z< m >)+ 7l )' 



m = 1,2 



such that 



\Ft } m\ < 

lim T*{(X?,X2,FW(X?) t FW(X2)) £ A n } = 1, 

n — >-oo 

limsup(£K(X 1 ", XJ, F^(X?), f£\x%))]~ 

n— >oo 

E{d n (X?,X2,z£\zW)}) <0, 



P 



lim sup E 



d n (x?,xz,hV(zU,zW 



<D, 



Vn > 1 



I(F^(X 1 y,F^(X 2 ))>I(Z^;Z 



72, 



where F^(X rn ) ^ {F^iX-)}^. 



Ri 



Ri > 1(X 1 
R 2 > 7(X 2 
R 2 >I(X 1 ; 



ZW)-I(ZW;Z<V), 
Z<V)-I(ZW;ZW), 
ZW)+I(X 2 ;ZW)-I(ZW;ZW). 



Remark 2 In Theorem^ the quantity J_(X; Y) denotes 
the spectral inf-mutual information rate defined by 

1 P XnY jX n ,Y n ) 
p- hm mf - In — — , 

n^oo n P Xn {X n )P Yn (Y n ) 



where 



p-liminf Z n = — p-limsup(— Z n ). 



The proof of Theorem^ is analogous to the proof of The- 
orem 1 in [7], and it relies heavily on Lemma\^ presented 
below, which is an easy consequence of Lemma\^ Note 
that the rate- distortion function for the Wyner-Ziv prob- 
lem with an average distortion criterion can be easily ob- 



(2) 

tained by letting Z n — X 2 and 
Ri 



1 



H(Xo) = p- lim sup — In ■ 



in Theorem^] Furthermore, Theorem^] can be easily ex- 
tended to the case of multiple terminals with one average 
distortion criterion and multiple maximum distortion cri- 
terions. 
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